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Situational  Awareness  in  Teams  with  Distributed  Expertise: 

A  Multi  ievel  Approach 

Background 

In  early  1990,  llgen  and  Hollenbeck  began  a  project  designed  to  study  decision 
making  in  hierarchical  teams  with  members  who  differed  in  expertise  and  status.  A 
literature  review  was  conducted  (llgen,  Major,  Hollenbeck,  &  Sego,  1993),  and  a  model 
of  team  decision  making  was  developed  based  on  an  adaptation  of  a  model  frequently 
used  to  study  individual's  decision  making  behavior,  the  Brunswik  Lens  Model  (llgen, 
Major,  Hollenbeck,  &  Sego,  1995).  This  model  was  used  to  develop  the  Multilevel 
Theory  of  team  decision  making. 

Concomitant  with  the  theoretical  work  mentioned  above,  a  team  decision  making 
simulation  was  constructed  which  allows  for  studying  the  behavior  of  four-person 
decision  making  teams.  The  simulation,  known  by  its  acronym,  TIDE^,  for  "team 
interactive  decision  exercise  for  teams  incorporating  distributed  expertise"  was  used  for 
most  of  the  initial  empirical  work  on  the  model.  Extensive  research  with  four-person 
teams  established  that  the  major  behavioral  constructs  of  interest  in  team  decision 
making  research  can  be  successfully  manipulated  and  studied  using  the  TIDE^  exercise 
(Hollenbeck,  Sego,  llgen.  Major,  Hedlund,  &  Phillips,  1997;  Hollenbeck,  llgen,  &  Tuttle, 
1995).  Data  analytic  methods  for  analyzing  multi-level  data  were  developed 
(Hollenbeck,  llgen  &  Sego,  1994)  and  results  employing  this  simulation  generally 
supported  the  major  tenets  of  the  multi-level  theory  (Hedlund,  llgen,  &  Hollenbeck, 

1998;  Hollenbeck,  llgen,  Sego,  Hedlund,  Major,  &  Phillips,  1995;  Hollenbeck,  llgen, 
LePine,  Colquitt,  &  Hedlund,  1998). 
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The  objectives  of  the  present  project  were  to  link  research  conducted  at  Michigan 
State  University  on  the  Multilevel  Theory  with  research  being  conducted  at  the 
Armstrong  Laboratories  at  Brooks  Air  Force  Base  on  team  decision  making  and 
situational  awareness.  The  first  five  months  of  this  project  (Aprii  1995  to  September 
1995)  were  dedicated  to  establishing  a  common  research  agenda,  sharing  technology, 
and  developing  standardized  tasks,  protocols  and  measures  for  future  studies.  The 
principal  investigators  at  Michigan  State  worked  closely  with  personnel  at  the  Armstrong 
Laboratory  including  Dr.  Samuel  Schiflett,  Ms.  Linda  Barrett,  Dr.  Douglas  Eddy  (NTI 
Inc.),  and  Mr.  Matt  Dalrymple  (SRL,  Inc.)  in  generating  a  multi-lab  collaborative  program 
of  research. 

The  result  of  this  joint  effort  was  an  agreement  to  study  four  factors  central  to  the 
Air  Force's  concerns  about  team  decision  making  accuracy  in  command  and  control 
teams.  These  four  factors  included  team  coordination,  team  gender  composition,  the 
nature  of  communication  media,  and  the  potential  disruptive  influence  of  communication 
breakdowns.  Of  primary  interest  to  the  project  was  the  potential  main  effect  and 
interaction  between  these  antecedents  and  team  performance  (e.g.,  how  the  type  of 
communication  media  influences  the  relationship  between  gender  composition  and 
team  performance  or  the  effect  of  communication  breakdowns  on  team  performance). 

In  addition,  this  program  of  research  was  to  use  the  Multilevel  Theory  of  team 
decision  making  as  a  framework  for  understanding  how  and  why  the  antecedents  (team 
coordination,  gender  composition,  media,  and  breakdowns)  impacted  decision  making 
accuracy.  According  to  the  Multilevel  Theory,  the  impact  that  these  antecedents  would 
have  on  overall  team  decision  making  accuracy  is  mediated  by  the  three  core 
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constructs  of  the  theory.  These  are  team  informity  (how  much  relevant  information  the 
team  can  bring  to  bear  on  the  decision),  staff  validity  (the  predictive  value  of  individual 
staff  members'  recommendations  to  the  team  leader)  and  hierarchical  sensitivity  (the 
degree  to  which  the  leader  can  accurately  weigh  the  different  opinions  of  staff 
members). 

The  remainder  of  the  report  presents  discussions  of  work  that  was  done  within 
specific  content  areas  addressed  by  this  grant. 

Sex  Composition  and  Computer  Mediated  Communication 

Literature  review.  The  initiai  empiricai  work  on  the  iikely  effects  of  sex 
composition  on  team  decisions  followed  an  extensive  literature  review  (Hollenbeck, 
llgen,  LePine,  &  Hedlund,  1995).  The  review  was  structured  around  an  analysis  of  team 
decision  making  requirements  from  the  perspective  of  the  multi-level  theory.  Once  sub¬ 
tasks  associated  with  team  decision  making  were  identified,  major  conceptual  and 
theoretical  approaches  to  sex  differences  were  reviewed  and  related  to  the  critical  sub¬ 
tasks  in  hierarchical  decision  making  teams.  It  was  argued  that  two  different  theoretical 
approaches,  evolutionary  theory  and  social  role  theory  showed  promise  for  improving 
understanding  of  the  impact  of  sex  composition  on  team  performance. 

Sex  composition  with  TIDE^  :  Computer  mediated  communication  only. 
Empirical  investigation  of  sex  composition  effects  were  begun  with  a  study  conducted 
by  the  Armstrong  Laboratories  research  team.  In  a  study  running  in  parallel  with  the 
Armstrong  study,  a  preliminary  examination  into  sex  composition  effects  was  performed 
at  Michigan  State  University.  An  experiment  was  run  with  49  teams  of  varied  sex 
composition  working  on  the  AWACs  version  of  TIDE^  where  there  was  no  voice 
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communication  between  team  members.  For  the  most  part,  these  49  teams  were 
replications  of  the  38  teams  run  at  Armstrong  Laboratories  under  similar  conditions. 

In  the  original  Armstrong  Laboratory  study,  focused  on  team  decision  making 
accuracy,  there  was  only  a  single  comparison  that  showed  a  statistically  significant 
difference  between  teams  that  was  attributable  to  sex  composition.  Specifically,  it  was 
found  the  all  male  teams  outperformed  teams  where  the  leader  was  male  but  the  staff 
was  all  female.  Since  there  were  six  different  configurations  of  teams  in  terms  of  sex 
composition,  this  allowed  for  15  specific  comparisons.  At  the  .05  level  of  statistical 
significance,  one  would  expect  at  least  one  significant  comparison  based  solely  upon 
chance.  Hence,  sampling  error  could  not  be  ruled  out  as  a  possible  alternative 
explanation  for  the  single  result  discovered. 

To  reduce  the  possibility  of  sampling  error  as  an  alternative  explanation,  the 
results  for  the  49  teams  run  at  Michigan  State  were  combined  with  the  38  run  at  Brooks. 
Although  sampling  error  was  a  potential  problem  in  both  these  data  sets  when 
examined  in  isolation,  the  combined  sample  of  77  teams  largely  obviated  sampling  error 
issues.  In  the  combined  sample,  no  significant  sex  composition  effects  were  found,  and 
although  only  1  out  of  15  effects  were  significant  in  the  MSU  sample  when  used  in 
isolation,  this  effect  was  not  the  same  as  that  found  in  the  Brooks  study.  Thus,  we 
concluded  that,  on  the  traditional  TIDE^  synthetic  task,  there  were  not  strong,  consistent 
effects  for  sex  composition. 

Because  there  was  no  opportunity  for  voice  communications  in  either  of  these 
studies,  there  were  severe  limits  on  both  the  frequency  and  content  of  communications 
between  team  members.  Since  communication  differences  are  one  of  the  more  likely 
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avenues  through  which  sex  differences  might  manifest  themseives,  the  traditional  TIDE^ 
task  may  not  have  been  conducive  to  finding  differences  attributable  to  sex  composition. 
Hence,  we  ran  a  second  study  that  allowed  voice  communications. 

Sex  Composition  and  Voice  Communications.  Examined  68  teams  of  varied 
sex  composition  who  worked  on  the  AWACs  version  of  TIDE^.  Ail  teams  communicated 
verbally  over  the  Telex  System,  were  recorded  with  the  Racal  System  and  these  data 
were  coded  using  categories  of  communication  derived  by  tailoring  general  categories 
to  the  task. 

Factor  analytic  results  indicated  that  four  factors  accounted  for  the  content  of 
communications  in  decision  making  teams.  These  factors  captured  the  degree  to  which 
the  team's  communications  focused  on  {^)  inputs  (e.g.,  discussing  the  information 
regarding  targets),  outputs  (e.g.,  discussing  recommendations  or  the  final  decision), 
process  (dealing  with  coordination  issues),  and  non-task  issues  (e.g.,  social 
communications). 

Regression  results  indicated  that  sex  composition  affected  two  aspects  of  the 
communication  content,  output  (R^  =  .10)  and  process  (R^  =  .29).  Communication 
content,  in  turn,  affected  two  core  constructs  of  the  Multilevel  Theory,  team  informity  (R 
=  .70)  and  hierarchical  sensitivity  (R^  =  .20).  Finally,  the  core  constructs  explained  a 
large  portion  of  the  variance  in  team  decision  making  accuracy  (R^  =  .39),  thus 
supporting  the  overall  model  that  sex  composition  affected  communication  content 
which,  in  turn,  affected  the  core  constructs  of  the  theory.  Finally  the  core  constructs 
affected  team  decision  making  accuracy.  In  each  case,  constructs  intervening  between 
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sex  composition  and  team  decision  making  accuracy  mediated  the  composition  effect, 
and  core  constructs  mediated  the  effect  of  communication  content  on  accuracy. 

The  nature  of  the  effects  suggested  that  all  male  teams  performed  worse  relative 
to  mixed  team  or  all  female  teams,  and  that  this  was  attributable  to  too  much  discussion 
of  process  as  opposed  to  input  and  output.  This  in  turn  harmed  their  overall 
performance  because  it  lowered  team  informity  and  hierarchical  sensitivity.  The  results 
from  this  study  are  currently  be  written  up  for  publication. 

Communication  Media 

A  study  of  the  effects  of  communications  media  employed  1 03  teams  that  were 
randomly  assigned  to  one  of  three  communication  media;  (a)  all  voice,  (b)  no  voice 
(computer  mediated)  or  (c)  strategically  mixed  communications.  The  strategic  condition 
based  medium  selection  on  the  memory  requirements  associated  with  the  message 
(e.g.,  long  term  versus  short  term  memory),  requirements  associated  with  speed  of 
transmission  (e.g.,  slow  versus  fast),  and  requirements  for  precision  (e.g.,  precision 
required  versus  not  required).  Thus,  in  the  strategically  mixed  condition,  electronic 
transmissions  were  used  for  raw  data  regarding  specific  targets  (high  precision  and 
speed,  but  no  long  term  memory  requirements),  voice  communications  were  used  for 
feelings  and  subjective  opinions  regarding  specific  targets  (low  precision  and  low 
memory  requirements,  but  high  need  for  speed),  and  written  text  messages  were  used 
for  task  coordination  messages  that  generalized  across  individual  targets  (high  memory 
and  precision  requirements,  but  low  speed  requirements). 

in  the  study  we  also  examined  the  degree  to  which,  the  nature  of  the 
communication  media  interacted  with  characteristics  of  the  team  members  (high  versus 
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low  ability)  and  the  task  (high  stress  versus  low  stress.  Thus,  testing  the  general  model 
depicted  as  follows: 


Results  indicated  that  teams  performing  in  the  strategically  mixed  media  condition 
outperform  voice  only  and  no  voice  teams  (R^  =  .07),  that  high  ability  teams 
(operationalized  in  terms  of  staff  validity)  outperformed  low  ability  teams  (R^  =  .29),  and 
that  team  member  ability  interacted  with  communication  medium  (R^  =  .09).  The  nature 
of  this  interaction  indicated  that  the  advantages  of  the  strategically  mixed 
communication  medium  were  particularly  pronounced  in  teams  with  low  or  varied  ability. 
Results  pertaining  to  stress,  are  still  pending,  and  require  a  within  team  (i.e.,  target  by 
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target  analysis)  that  is  preceding  now.  The  results  from  this  study  is  being  written  up  for 
publication  at  this  time. 

Communication  Breakdowns 

This  study  used  the  Multilevel  Theory  to  analyze  the  implications  of 
communication  breakdowns  in  the  area  of  team  decision  making.  |t  employed  143 
teams  that  worked  on  the  TIDE^  task  and  experienced  communication  failures  during 
the  course  of  a  two  hour  experimental  session.  The  primary  outcome  studied  was  the 
degree  to  which  communication  failures  lead  to  inaccurate  decision  making,  and  the 
mechanisms  (in  terms  of  the  core  constructs  of  the  Multilevel  Theory)  through  which  this 
took  place.  This  study  also  examined  the  degree  to  which  characteristics  of  the  team 
members  such  as  cognitive  ability  (LePine,  Hollenbeck,  &  llgen,  1997),  characteristics 
of  the  task  (goals  and  feedback),  and  characteristics  of  the  communication  failure 
(gradual  versus  abrupt)  moderated  the  negative  impact  of  communication  failures. 

There  were  several  noteworthy  results  from  this  study.  First,  teams  characterized 
as  having  higher  levels  of  cognitive  resources  had  higher  odds  of  successfully  working 
around  (i.e.,  adapting  to)  the  communication  breakdown.  Second,  there  was  also  a 
relationship  between  the  nature  of  the  breakdown  and  adaptation.  Teams  were  more 
likely  to  detect  and  successfully  overcome  an  abrupt  breakdown  relative  to  a  gradual 
one.  Finally,  the  nature  of  the  team’s  performance  relative  to  the  team's  goals  was  also 
a  significant  factor  affecting  the  response  to  the  breakdown.  Specifically,  teams  who 
were  performing  below  their  goal  levels  were  less  likely  to  successfully  adjust  to  the 
communication  breakdown  relative  to  teams  that  were  performing  above  their  goal 
levels.  Performing  above  their  goal  levels  provided  teams  with  some  degree  of 
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psychological  "slack"  that  could  be  used  for  trial  and  error  experimentation  in  working 
around  the  breakdown.  Teams  that  were  performing  below  their  goal  levels  tended  to 
restrict  their  focus  and  were  unable  to  adapt  well  to  the  communication  failure  (LePine, 
Hollenbeck,  &  llgen,  1998). 

Accuracy  Decomposition 

One  important  external  development  that  occurred  while  we  were  working  on  this 
grant  was  the  publication  of  Gigone  and  Hastie's  (1997)  article  on  accuracy 
decomposition.  In  this  article,  Gigone  and  Hastie  showed  how  conventional  measures  of 
decision  accuracy  could  be  decomposed  into  three  distinct  components:  linear 
consistency,  mean  bias  and  variability  bias.  Gigone  and  Hastie  argued  that,  given  the 
relative  independence  of  these  three  facets,  researchers  should  be  developing  theories 
for  each  of  the  different  forms  of  accuracy  rather  than  or  in  addition  to  theories  of  overall 
accuracy  criterion  since  the  components  may  not  all  have  similar  causes. 

We  conceptually  analyzed  the  Multilevel  Theory  against  the  three  facets  of 
accuracy  isolated  by  Gigone  and  Hastie  (1997),  and  there  seemed  to  be  a  number  of 
reasons  to  believe  that  this  theory  is  primarily  a  theory  of  linear  consistency,  rather  than 
mean  bias  or  variability  bias.  We  then  empirically  tested  the  generalizability  of  the 
Multilevel  Theory  across  these  three  different  facets  of  accuracy  with  an  existing  data 
set  that  included  105  teams  who  worked  on  the  TIDE^  task.  These  data  are  reported  in 
Hollenbeck,  Colquitt,  llgen,  LePine,  and  Hedlund  (1998). 

The  results  of  this  study  indicated  that  the  core  constructs  of  the  Multilevel 
Theory  accounted  for  63%  of  the  variance  in  linear  consistency,  20%  of  the  variance  in 
variability  bias,  but  none  of  the  variance  in  mean  bias.  We  concluded  that  the  Multilevel 
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Theory  is  primarily  a  theory  of  linear  consistency,  and  that  alternative  theories  needed 
to  be  developed  to  deal  with  other  facets  of  accuracy,  especially  mean  bias. 

Mean  Bias  Study 

Criterion  decomposition  is  a  technique  that  attempts  to  improve  a  theory's  ability 
to  predict  a  criterion  by  adding  variables  that  are  specifically  aimed  at  sub-components 
of  the  criterion  least  well  predicted  by  the  theory.  The  purpose  of  the  mean  bias  study 
was  to  use  this  approach  in  attempting  to  extend  the  Multilevel  Theory  (MLT)  of  team 
decision  making.  Specifically,  this  study  examined  one  individual  difference  variable 
(team  member  aggressiveness)  and  one  situational  factor  (direct  feedback  on 
levels  of  mean  bias)  that  might  be  expected  to  affect  mean  bias,  and  tested  the  degree 
to  which  these  factors  improve  the  MLTs  ability  to  predict  overall  decision  making 
accuracy.  As  such,  this  was  the  first  study  in  the  area  of  decision  making  accuracy  that 
employed  mean  bias  as  the  primary  dependent  variable,  and  thus,  provides  the  first 
evidence  regarding  factors  that  might  predict  this  component  of  overall  team  decision 
making  accuracy. 

Consistent  with  past  research  (Hollenbeck,  Colquitt  et  al.,  1998),  results  from  this 
study  of  320  individuals  in  80  four-person  teams  confirmed  that  the  MLT  predicts  overall 
decision  accuracy  primarily  because  it  is  a  theory  of  linear  consistency.  The  core 
constructs  of  the  MLT  explained  almost  no  variance  in  mean  bias.  The  best  way  to 
improve  this  theory,  therefore,  is  to  add  variables  to  it  that  would  be  predictive  of  mean 
bias-a  component  of  overall  accuracy  that  is  independent  of  linear  consistency.  This 
general  proposition  was  supported  in  that  staff  member  aggressiveness  and  feedback 
both  had  direct  effects  on  mean  bias.  These  two  variables  also  interacted  in  the  sense 
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that  feedback  on  mean  bias  neutralized  the  effect  of  staff  aggressiveness.  Consistent 
with  expectations,  when  staff  aggressiveness  and  mean  bias  feedback  were  added  to 
the  core  constructs  of  the  MLT,  this  enhanced  the  amount  of  variance  explained  in 
overall  team  decision  making  accuracy  by  on  mean  bias  improved  the  prediction  of 
overall  team  decision  accuracy  over  and  above  that  predicted  by  the  core  constructs  of 
the  MLT  alone  by  9%.  This  study  has  been  written  up  and  was  recently  submitted  for 
publication  (LePine,  Hollenbeck,  ligen,  Colquitt,  &  Ellis,  1998). 

Summary  and  Conclusion 

Small  decision  making  teams  with  leaders  and  staff  members  whose  areas  of 
expertise  overlap  but  also  differ  in  several  respects  continued  to  be  the  focus  of  this 
research  effort.  As  has  been  the  case  with  our  work  in  the  past,  critical  issues  related  to 
the  quality  of  decisions  made  by  such  teams  are  identified  both  from  an  understanding 
of  the  types  of  problems  faced  by  teams  in  the  field  and  by  theoretical  issues  raised  by 
us  and  other  researchers  studying  teams.  These  issues  are  then  addressed  in  the 
laboratory  setting  using  three  or  four  person  teams  trained  on  a  decision  making 
simulation.  They  perform  in  that  setting  for  two  to  three  hours. 

The  present  research  effort  began  by  addressing  conditions  likely  to  influence  the 
quality  of  decisions  in  teams  of  the  type  described  above.  The  conditions  of  interest 
were  team  composition  and  communication  mode.  Composition  focused  on  gender  for 
two  reasons.  First,  at  a  time  when  integration  of  the  sexes  across  most  all  job 
classifications  is  both  a  social  and  a  legal  imperative,  decision  making  teams  are  and 
must  continue  to  be  composed  of  both  males  and  females.  Yet,  past  research  has 
found  that  the  composition  of  teams  can  affect  team  processes  and  outcomes. 
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Research  on  this  grant  using  decision  making  teams  found  both  process  and  outcome 
effects.  The  data  indicated  that  all  male  teams  were  more  likely  than  mixed  or  all 
female  teams  to  spend  greater  amounts  of  time  and  effort  discussing  the  process  by 
which  they  should  function  as  a  team  than  were  mixed  gender  or  all  female  teams,  and 
that  greater  amounts  of  time  negatively  impacted  on  team  decision  making  accuracy. 
The  research  points  to  the  need  to  consider  gender  composition  from  a  more  complex 
standpoint  than  simply  asking  if  it  affects  performance.  Rather,  there  is  a  need  to 
understand  the  task  from  the  standpoint  of  a  clear  task  analysis  and  then  ask  what  team 
processes  in  the  particular  task  setting  are  and  are  not  likely  to  be  affected  by  gender 
composition  and  is  the  composition  likely  to  affect  performance? 

Other  conditions  investigated  with  respect  to  team  decision  making  also  began 
with  the  assumption  that  the  task  demands  serve  as  a  basis  for  looking  at  the  effects  of 
the  conditions  on  team  decision  making.  This  work  focused  on  extending  previous 
research  to  include  typical  means  of  communication  open  to  small  decision  making 
teams.  In  particular,  we  employed  both  communications  sent  through  computer 
terminals  as  well  as  voice  communications.  In  some  cases,  teams  were  restricted  to 
one  or  the  other  form  of  communication.  In  other  cases,  we  first  analyzed  the  task  from 
the  standpoint  of  the  demands  that  the  form  of  information  needed  to  do  the  task  made 
on  communications.  For  example,  when  the  information  needed  to  be  precise  and 
stored  in  some  way  that  was  easily  retrievable  without  a  bias  being  introduced  between 
the  time  that  the  message  was  originally  received  and  the  time  when  it  was  needed, 
information  was  sent  and  stored  electronically.  On  the  other  hand,  when  the  information 
was  somewhat  ambiguous  and  needed  to  be  refined  and/or  agreement  was  needed 
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among  team  members  to  be  able  to  deal  with  the  information,  communicating  by  voice 
seemed,  a  priori,  to  be  a  preferred  mode  of  communication.  The  data  confirmed  that  a 
mixed  mode  communication  system  where  the  mode  was  dictated  by  the  information 
demands  of  the  task  led  to  better  decisions  than  either  mode  in  its  pure  form.  It  is 
suggested  that  future  research  needs  to  explore  ways  to  refine  the  ways  in  which  team 
decision  making  tasks  are  analyzed  as  to  their  communication  demands,  then  the 
communication  mode  can  be  selected  more  strategically  to  obtain  a  better  fit  to  the 
communication  demands  of  the  subtasks  faced  by  the  teams. 

Finally  refinement  of  the  initial  model  of  team  decision  accuracy  (see  Hollenbeck, 
llgen,  Sego  et  al.,  1995)  was  the  focus  of  two  studies.  First,  following  the  publication  of 
Gigone  and  Hastie’s  (1997)  Psychological  Bulletin  article  on  decomposing  constructs 
like  our  team  decision  accuracy  measure,  team  decision  data  were  analyzed 
decomposing  our  accuracy  measure  into  the  three  components.  These  data  showed 
that  when  teams  were  inaccurate  in  our  setting,  the  three  components  of  our  model, 
team  informity,  staff  validity,  and  hierarchical  sensitivity,  were  able  to  predict  variability 
bias  quite  well.  Mean  bias  was  poorly  predicted.  A  second  study  focused  in  on  mean 
bias  by  seeing  whether  feedback  given  to  team  members  and  an  individual  difference 
critical  for  the  type  of  simulation  being  used  for  the  decision  making  task  would  improve 
our  ability  to  predict  mean  bias.  The  results  showed  that  adding  additional  predictors 
and  their  interactions  did  improve  prediction  of  mean  bias  in  the  teams.  Thus,  the  work 
offered  information  useful  for  refining  the  prediction  of  team  decision  accuracy  and  its 
components.  As  with  all  the  studies  undertaken  in  this  research  effort,  attention  was 
directed  toward  improving  understanding  of  factors  that  influence  the  ability  of  teams  to 
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make  accurate  decisions  when  those  teams  have  a  leader  responsible  for  their  decision 
and  staff  members  who  have  expertise  that  overlaps  but  is  not  totally  redundant.  In  all 
cases,  the  multi-level  theory  was  found  to  be  a  useful  guide  for  the  research. 
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